Multispectral/hyperspectral medical instrument

ABSTRACT

A medical instrument that comprises: a first-stage optic responsive to a tissue surface of a patient; a spectral separator optically responsive to the first stage optic and having a control input; an imaging sensor optically responsive to the spectral separator and having an image data output; and a diagnostic processor having an image acquisition interface with an input responsive to the imaging sensor and a filter control interface having a control output provided to the control input of the spectral separator.

CROSS-REFERENCES TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 13/303,107, filed Nov. 22, 2011, which is a continuation ofU.S. patent application Ser. No. 11/165,587, filed on Jun. 24, 2005, nowissued as U.S. Pat. No. 8,175,688, which is a divisional of U.S. patentapplication Ser. No. 09/182,898, filed on Oct. 30, 1998, now issued asU.S. Pat. No. 6,937,885, which claims the benefit of U.S. ProvisionalPatent Application, Ser. No. 60/093,041, filed on Nov. 10, 1997, allwhich are hereby incorporated by reference in their entirety.

FIELD OF THE INVENTION

The invention relates to a surgical and diagnostic instrument forperforming real-time general-purpose imaging during surgery, clinicalprocedures, or other medical evaluations.

BACKGROUND OF THE INVENTION

Spectroscopic imaging devices which employ Acousto-Optic Tunable Filters(AOTF), Liquid Crystal Tunable Filters (LCTF), or dispersive gratingsare known. Such devices have been used for microscopy and remotesensing.

SUMMARY OF THE INVENTION

Generally, the invention features a medical instrument that includes anoptic responsive to a surface of tissue of a patient, a spectralseparator optically responsive to the optic, and an imaging sensoroptically responsive to the spectral separator. The instrument alsoincludes a diagnostic processor having an image acquisition interfaceresponsive to the imaging sensor and a filter control interface to whichthe spectral separator is responsive.

The spectral separator can be a tunable filter, such as a liquid crystaltunable filter, and the imaging sensor can be a two-dimensional imagingarray, such as a charge coupled device. The optic can include a macrolens, an adjustable lens, or a probe that includes an imaging fiberoptic cable and a stand can be connected relative to the optic toposition the optic relative to the patient. The control interface can beoperable to adjust the filter at least twenty times to acquirehyperspectral data for redisplay in real time. The medical instrumentcan perform diagnostic processing for images acquired exclusively undervisible light.

The diagnostic processor can also include a general-purpose processingmodule and diagnostic protocol modules, which can each include filtertransfer functions and an image processing protocol. The general-purposeprocessing module can be operative to instruct the filter tosuccessively apply the filter transfer functions to light collected fromthe patient, to acquire from the imaging sensor a number of images ofthe collected light each obtained after one of the filter transferfunction is applied, and to process the acquired images according to theimage processing protocol to obtain a processed display image. Thegeneral-purpose processor can be a real-time processor operative togenerate a processed display image within a time period on the order ofthe persistence of human vision. It may also be operative to acquiresome images more slowly depending on the number of wavelengths andcomplexity of diagnostic processing protocol. The sensor and filter canbe operative in the visible, infra-red, and UV regions.

Instruments according to the invention are advantageous in that they canpermit a surgeon or a physician to diagnose a medical condition ordevelop a surgical strategy based on real-time images obtained duringsurgery or in the course of performing clinical procedures or othermedical evaluations. The physician may therefore be able to obtainsignificantly more information about a patient's condition than he mightotherwise have been able to assemble by presenting an interactiveinterface. This additional information may permit a given surgicalprocedure to be carried out more precisely and may lead to moresuccessful surgical results. It may also enhance the precision andresults of other types of medical evaluations and procedures.

The general-purpose nature of the instrument can also help the surgeondevelop significant amounts of medical information in time-criticalsurgical situations. For example, a patient may undergo relativelystraight-forward surgery during which the surgeon may discover a tumoror another internal condition. With an instrument according to theinvention, the physician can spend a small amount of additional timewith the patient under anesthesia and determine the nature and extent ofthe tumor. This can be particularly beneficial during major surgery,where extending surgery duration poses a potential morbidity andmortality risk. Because the procedure is rapid and noninvasive, thepatient is exposed to little additional risk. The benefit of immediatediagnosis and evaluation is significant.

An instrument according to the invention may also be able to provide awide variety of diagnostic capabilities, allowing a physician to enhancethe capabilities of his or her practice substantially in a variety ofdifferent realms, without investing in a number of instruments. Thephysician can then enhance or update the instrument by the addingsoftware modules that are specifically targeted towards certainconditions of particular tissues, subsystems, or disease states. Thiscan allow a single base instrument to be configured for a variety ofdifferent types of practices, and priced according to the type ofpractice to be served by the instrument. For example, a general-purposeinstrument to be used by a general surgeon could include a package ofdiagnostic protocols that would permit the diagnosis of a variety ofconditions that a general surgeon might encounter, while aneurosurgeon's module might be added to allow a specialist to detectparticular conditions within the brain. Electronic and optical upgradesmay also be provided to update, specialize, or improve the performanceof the instrument. Such upgrades can include processing modules, memoryboards, lenses, and the like.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective diagram of a macroscopic instrument according tothe invention;

FIG. 2 is a perspective diagram of a rigid or flexible probe-basedinstrument according to the invention;

FIG. 3 is a block diagram of the instrument of FIG. 1; and

FIG. 4 is a flowchart illustrating the operation of the system of FIG.1.

DETAILED DESCRIPTION OF AN ILLUSTRATIVE EMBODIMENT

Referring to FIG. 1, an instrument according to the invention 10includes an imaging module 12 mounted on a surgical stand 14. In thisembodiment, the surgeon can direct the imaging portion 12 towards apatient 20 by manipulating a control 16 that adjusts the attitude of theimaging portion through a positioning mechanism 18.

Referring to FIG. 2, an alternative embodiment of the invention 22 mayinclude a probe such as a rigid or flexible endoscopic, thoracoscopic,laproscopic, or angioscopic probe 24 connected to an imaging station 30via a fiber optic cable 26. The surgeon can manipulate the probe withinthe patient in a minimally-invasive surgical procedure and derive imagesfrom a portion of the patient and display these images on a display 28.A medical implement 32, such as a laser, can also be provided throughthe probe. For example, after diagnosing a particular condition, aphysician can begin laser ablation therapy to remedy it.

Referring to FIG. 3, a medical instrument according to the invention 34may include an optical acquisition system 36 and a diagnostic processor38. The acquisition system 36 includes a first-stage imaging optic 40, aLiquid-Crystal Tunable Filter (LCTF) 42, a second-stage optic 44, and animaging element 46. The first-stage optic receives light collected fromthe patient and focuses it onto the surface of the LCTF. The first-stageoptic can be a simple or compound macro lens in the case of amacroscopic instrument (FIG. 1). In a probe-based instrument (FIG. 2),the first stage optic can include imaging optics within a probe such asa endoscopic, laproscopic, thoracoscopic, or angioscopic probe. Thefirst stage lens can also be adjustable, allowing a physician to scanlarger areas of tissue and then zoom into particular regions.

The LCTF 42 is a programmable filter that filters out all but awavelength region of interest from the light collected from the patient.The second-stage optic 44 receives the remaining light from the LCTF andfocuses it onto the image sensor 46. The image sensor is preferably,although not necessarily, a two-dimensional array sensor, such as acharge-coupled device array (CCD), which delivers an image signal to thediagnostic processor 38.

The diagnostic processor 38 includes an image acquisition interface 50,that has an input responsive to an output of the image sensor 46 and anoutput provided to a general-purpose operating module 54. Thegeneral-purpose operating module includes routines that perform imageprocessing, and that operate and control the various parts of thesystem. It has a control output provided to a filter control interface52, which in turn has an output provided to the LCTF 42. Thegeneral-purpose operating module also interacts with a number ofdiagnostic protocol modules 56A, 56B, . . . , 54N, and has an outputprovided to the video display 12. The diagnostic processor can includespecial purpose hardware, general-purpose hardware with special-purposesoftware, or a combination of the two. The diagnostic processor alsoincludes an input device 58, which is operatively connected to thegeneral-purpose operating module. A storage device 60, and a printer arealso operatively connected to the general-purpose operating module.

In operation, referring to FIGS. 3 and 4, a surgeon employing theinstrument begins by selecting a diagnostic protocol module using theinstrument's input device (step 100). Each diagnostic protocol module isadapted to detect particular characteristics of the surface of one ormore types of tissue. For example, the surgeon might select a modulewhich enhances the visibility of cancerous tissue. The surgeon wouldthen direct the camera at the area of interest and begin inspecting iteither under ambient light or with the aid of a supplemental lightsource, which can be filtered to emphasize particular specialcharacteristics of the light it emits.

The diagnostic processor 38 responds to the surgeon's input by obtaininga series of filter transfer functions and an image processing protocolfrom the selected diagnostic protocol module 56. The diagnosticprocessor provides the filtering transfer functions to the LCTF 42 viaits filter control interface 52 (step 102) and then instructs the imageacquisition interface 50 to acquire and store the resulting filteredimage from the image sensor 46 (step 104). The general-purpose operatingmodule 54 repeats these filtering and acquiring steps one or more times,depending on the number of filter transfer functions stored in theselected diagnostic protocol module (see step 106). The filteringtransfer functions can represent bandpass, multiple bandpass, or otherfilter characteristics.

Once the image acquisition interface 50 has stored images for all of theimage planes specified by the diagnostic protocol chosen by the surgeon,it begins processing these image planes based on the image processingprotocol from the selected diagnostic protocol module 56N (step 108).Processing operations can include general image processing of combinedimages, such as comparing the relative amplitude of the collected lightat different wavelengths, adding amplitudes of the collected light atdifferent wavelengths, or computing other combinations of signalscorresponding to the acquired planes. The processing operations can alsoinclude more complex multivariate statistical techniques to compute theimage (e.g., chemometrics). The computed image is displayed on thedisplay 12. It can also be stored in the storage device 60 or printedout on the printer 62.

The processing operations can also be based on a diagnostic knowledgebase. This database can include data resulting from the comparisonbetween optical and actual diagnoses. Each instrument can alsocontinuously update its database as it is used to perform diagnoses,thereby steadily expanding its diagnostic capabilities.

In order to provide a real-time or near-real-time image to the surgeon,the instrument repeatedly acquires planes and processes them to developan image to be displayed to the surgeon. This allows the surgeon to movethe instrument, or to view moving organs, such as a beating heart. Thisconstant acquisition and processing continues until the surgeon eitherturns the instrument off (step 110) or selects a different imaging mode(step 112). The diagnostic processor 38 preferably has sufficientprocessing power to update the screen in this way at video rates (i.e.,about 30 frames per second), although rates as low as a few frames persecond may work quite well, and rates as low as one frame per minute maybe adequate for many purposes. On slower instruments, general lock-inschemes or other tracking modalities, such as cardiac gating, can beused to remove motion artifacts due to breathing or heart beat. Framerate may also be variable, depending on the number of wavelengths andthe complexity of the diagnostic procedure.

Preferably, the instrument can operate in multispectral, andhyperspectral, or even ultraspectral imaging modes. Multispectral modesinvolve image processing derived from a relatively small number ofspectral image planes (two wavelengths to about twenty wavelengths).Hyperspectral and ultra spectral imaging modes involve at least twentyimage planes and can produce significantly more accurate and informativeresults. Ultraspectral modes involve hundreds of wavelengths, and may beable to produce even further information about the patient.Hyperspectral and ultraspectral imaging may include selecting specificwavelength bands for discrimination of a particular diseased states, orit may also allow the instrument to scan for multiple conditions at thesame time.

It is also contemplated that both types of instrument can operate inconnection with an excitation source, such as an ultraviolet lamp and IRsource, or other means of spectral illumination or a laser to enhancethe received images. Although such excitation may not be necessary, itmay allow for the examination of different optical phenomenon andprovide additional diagnostic information. And both emission andreflectance modes can be combined in a diagnostic procedure eithersimultaneously or sequentially. Relative utilization of differentemission or reflection measurements involved in the same diagnosticprocedure can be obtained by modulating the different sources. Theinstrument can also develop light from bioluminescent sources introducedinto the patient.

Instruments according to the invention can also operate to processimages from image planes acquired at wavelengths outside of the visibleregion. In one particular embodiment, the instrument is sensitive to thevisible and near infra-red regions. It is also contemplated that farinfra-red be included to allow the instrument to sensemolecular-specific rotational modes.

An example of operation would include the use of a diagnostic protocolmodule that examined a first wavelength of about 550 and a secondwavelength of about 575 associated with oxy- and deoxy-hemoglobin todetermine blood oxygenation. The relationship between these wavelengthsis described in “Hemoglobin: Molecular Genetics and Clinical Aspects,”by H. Franklin Bunn and Bernard Forget, W. B. Sanders, 1986. Anotherexample would include the use of a diagnostic protocol module forexamining the Fourier transform infra-red spectra of the colon andrectum as described in “Human Colorectal Cancers Display AbnormalFourier Transform Spectra,” by Basil Rigas et al., Proceedings of theNational Academy of Science, pp. 8140-8144, 1987.

Surgical and medical applications of instruments according to theinvention can include, but are not limited to, determining tissueviability (i.e. whether tissue is dead or living tissue and whether itis predicted to remain living), detecting tissue ischemia (e.g., inheart, or in leg after a gunshot wound, differentiating between normaland malignant cells and tissues (e.g., delineating tumors, dysplasiasand precancerous tissue, detecting metastasis), differentiating betweenof infected and normal (but inflamed) tissue (e.g., extent of aorticroot infection), quantification and identification of pathogens, (e.g.,bacterial count of burn wounds and differentiating and delineating otherpathologic states. Application can also include tissue, blood chemistry,and blood flow (including oxy- and deoxyhemoglobin, myoglobindeoxymyoglobin, cytochrome, pH, glucose, calcium and other elements orbiological compounds alone or in combination). The instrument can alsobe applied by veterinarians to animals and by dentists to dentalapplications, such as peridental disease.

The present invention has now been described in connection with a numberof specific embodiments thereof. However, numerous modifications whichare contemplated as falling within the scope of the present inventionshould now be apparent to those skilled in the art. Therefore, it isintended that the scope of the present invention be limited only by thescope of the claims appended hereto. In addition, the order ofpresentation of the claims should not be construed to limit the scope ofany particular term in the claims.

The invention claimed is:
 1. A multispectral or hyperspectral medical instrument comprising: (A) an optical acquisition system (36) comprising an image sensor (46) that is configured to acquire multispectral, hyperspectral, or ultraspectral images; and (B) a diagnostic processor (38) comprising: (i) a general-purpose operating module (54) having output to a display, (ii) an image acquisition interface (50) having an input responsive to the output of the image sensor (46) and further having an image acquisition interface output that is provided to the general-purpose processing module, and (iii) instructions for applying any one of a plurality of diagnostic protocol modules (56) to the image acquisition interface output provided from the image acquisition interface (50), wherein each respective diagnostic protocol module in the plurality of diagnostic protocol modules includes instructions to acquire a plurality of image planes using optical acquisition system (36) and an image processing protocol with instructions to process the plurality of image planes to enhance visualization of a particular characteristic of the surface of one or more types of tissue, through the application of filtering and acquiring instructions associated with the respective diagnostic protocol.
 2. The multispectral or hyperspectral medical instrument of claim 1, wherein the optical acquisition system (36) comprises a first-stage imaging optic (40) in optical communication with the image sensor (46), the first-stage imaging optic including an adjustable lens.
 3. The multispectral or hyperspectral medical instrument of claim 1, wherein the image sensor (46) is a two dimensional imaging array.
 4. The multispectral or hyperspectral medical instrument of claim 1, wherein the image sensor (46) comprises a charge coupled device (CCD).
 5. The multispectral or hyperspectral medical instrument of claim 1, wherein the image sensor (46) is sensitive to the visible region of the electromagnetic spectrum.
 6. The multispectral or hyperspectral medical instrument of claim 1, wherein the image sensor (46) is sensitive to the near infra-red region of the electromagnetic spectrum.
 7. The multispectral or hyperspectral medical instrument of claim 1, wherein the image sensor (46) is sensitive to the visible and near infra-red regions of the electromagnetic spectrum.
 8. The multispectral or hyperspectral medical instrument of claim 1, wherein the image sensor (46) is sensitive to the far infra-red region of the electromagnetic spectrum.
 9. The multispectral or hyperspectral medical instrument of claim 1, wherein the general-purpose operating module (54) is operative to generate a processed display image between one time a second and thirty times a second.
 10. The multispectral or hyperspectral medical instrument of claim 1, wherein the diagnostic processor (38) is modular and upgradeable by adding additional diagnostic protocol modules (56), thereby expanding diagnostic capabilities.
 11. The multispectral or hyperspectral medical instrument of claim 1, wherein the diagnostic processor (38) further comprises a diagnostic database including data resulting from comparison of collected multispectral or hyperspectral images and actual diagnoses.
 12. The multispectral or hyperspectral medical instrument of claim 11, wherein the diagnostic database is updatable upon use by a clinician, thereby expanding a diagnostic capability of the diagnostic processor (38).
 13. The multispectral or hyperspectral medical instrument of claim 1, wherein each respective diagnostic protocol module in the plurality of diagnostic protocol modules (56) further includes instructions to process a plurality of images acquired by the image sensor (46) to obtain a multispectral or hyperspectral image.
 14. The multispectral or hyperspectral medical instrument of claim 13, wherein the processing of the plurality of images acquired by the image sensor (46) includes the use of multivariate statistical techniques.
 15. The multispectral or hyperspectral medical instrument of claim 1, wherein the particular characteristic of the surface of one or more types of tissue is the viability of a tissue.
 16. The multispectral or hyperspectral medical instrument of claim 1, wherein the particular characteristic of the surface of one or more types of tissue is tissue ischemia.
 17. The multispectral or hyperspectral medical instrument of claim 1, wherein the particular characteristic of the surface of one or more types of tissue is a malignant cell or tissue.
 18. The multispectral or hyperspectral medical instrument of claim 1, wherein the particular characteristic of the surface of one or more types of tissue is oxyhemoglobin and deoxyhemoglobin.
 19. The multispectral or hyperspectral medical instrument of claim 1, further comprising a probe (24) connected to the optical acquisition system (36) via a fiber-optic cable (26).
 20. The multispectral or hyperspectral medical instrument of claim 19, wherein the probe (24) is a rigid endoscopic probe, a flexible endoscopic probe, a thoracoscopic probe, a laproscopic probe, or an angioscopic probe. 